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Abstract 

In this paper we explore the decision regions of Linear Programming (LP) decoding. We compare 
the decision regions of an LP decoder, a Belief Propagation (BP) decoder and the optimal Maximum 
Likelihood (ML) decoder. We study the effect of minimal-weight pseudocodewords on LP decoding. We 

(N : 

;> present global optimization as a method for finding the minimal pseudoweight of a given code as well 

t-H ■ 

■ as the number of minimal-weight generators. We present a complete pseudoweight distribution for the 



[24, 12, 8] extended Golay code, and provide justifications of why the pseudoweight distribution alone 



ly-j ' cannot be used for obtaining a tight upper bound on the error probability. 

o ' 

Index Terms 

Belief propagation, bounded distance decoding, generators, global optimization, linear programming, 
t LP relaxation, pseudocodewords, pseudoweight. 



I. Introduction 

The behavior of the BP [1J decoder for the case of finite-length codes does not have simple 
characteristics, and can be very hard to predict. Linear programming is a well- studied discipline 
that provides efficient analysis tools. The relationship between linear programming decoding 
and belief propagation decoding was observed and characterized J2|, and the decision regions 
of these decoders are suggested to be tightly related. 
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The LP decoder receives the channel likelihood ratios which define an objective function, 
for which it finds an optimal solution that satisfies a set of constraints. These constraints are 
inequalities arisen from a given parity check matrix and form a polytope, also known as the 
fundamental polytope [3]. The fundamental polytope is a relaxation of the codewords polytope. It 
has a clear geometrical representation which is well-suited for finite-length analysis. The vertices 
of the fundamental polytope are every codeword, but also some non-codewords pseudocodewords 
JH. The fundamental cone 01 is the conic hull of the fundamental polytope. It has a vertex in 
the origin, and its edges are also referred to as minimal pseudo-codewords or generators flU. 
The fundamental cone has a more compact representation than the fundamental polytope, and it 
is sufficient to consider the fundamental cone for evaluating the performance of the LP decoder 

mi. 

The output of the LP decoder is always a vertex of the polytope which maximizes the 
channel likelihood ratios function. One of the most appealing properties of the LP decoder is 
the ML certificate property - whenever it returns an integral solution, the solution is guaranteed 
to be the ML codeword; otherwise an error is invoked. There are rare cases flfD for which the 
vertices of the polytope are codewords only, and in these cases the output of the LP decoder is 
identical to the output of the ML decoder. In these rare cases a polynomial-time ML decoding 
is attainable. However, for most cases, and when applied to good error-correcting codes, the LP 
decoder will suffer from decoding failures due to the presence of pseudocodewords. 

The minimal pseudoweight 01 of a pseudocodeword in LP decoding is the appropriate analog 
of the minimal Hamming weight in ML decoding. Furthermore, the minimum Hamming weight 
is known to be lower bounded by the minimal pseudoweight [7j. There are cases where the 
minimal pseudoweight equals the minimal Hamming weight, and in these cases, the existence 
of pseudocodewords may have a minor or even a negligible effect on the decoder's optimality. 

High Density Parity Check (HDPC) codes are characterized by a dense parity check matrix. 
Linear classical codes have a dense parity check matrix by design, which makes them less 
suitable for LP decoding. The denser the parity check matrix is the more vertices the fundamental 
polytope will have. Keeping in mind that the number of codewords of a given code is constant, 
one can realize that increasing the number of vertices is equivalent to increasing the number of 
pseudocodewords which are not codewords. 
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The BP algorithm is often in use for decoding low-density parity-check (LDPC) codes, for 
which it has both low complexity and good performance. Its low complexity is achieved due 
to the fact that the algorithm operates locally on a so-called Tanner graph (HI representation 
of the parity-check matrix. However, operating locally leads to a fundamental weakness of the 
algorithm - it may fail to converge due to non-codewords pseudocodewords. These pseudocode- 
words are valid assignments of the computation tree (H of the given code and decoder. Kelley 
and Sridhara [9] have proved that the pseudocodewords of the computation tree are a superset 
of the pseudocodewords which lie in the aforementioned fundamental polytope. 

The decision regions of a decoding algorithm provide a visualization of the decoder's decisions 
upon receiving channel signals. They provide a better intuition of the decoder operation, and 
can be used for comparing different decoding algorithms. The existence of pseudocodewords 
in iterative decoding and their effect on the decision regions were studied in [0, ifTOl . In the 
present work we examine the effect of pseudocodewords on the decision regions and on the 
performance of the LP decoder. 

Presenting a complete picture of the decision regions is usually impossible even for short- 
length codes, due to the number of dimensions involved in each decoded signal. Nevertheless, 
performing cuts in the signal space can provide a clear picture of specific decision regions, which 
illustrate the effect of pseudocodewords on the performance of the BP and LP decoders. 

In order to illustrate the different decision regions of the BP, LP and ML decoders, the [8, 4, 4] 
extended Hamming code was chosen. It is known that both BP and LP decoders are affected 
by the selection of the parity check matrix, therefore three different representations for the 
aforementioned code were chosen. 

The contribution of this paper is in providing a better understanding of the LP decoder 
operation. Global optimization is proposed as a method for finding the minimal weight generators 
of a given parity check matrix. The paper also presents the difficulties in obtaining a tight LP 
union bound, based on the generators' weight distribution, and explores the effect of minimal- 
weight pseudocodewords on the decision regions of the LP decoder. 

The rest of the paper is organized as follows: We provide some background on decision 
regions and a method to produce the decision regions in Section HO In Section [III] we present 
the different effect of minimal-weight pseudocodewords on LP decoding. A global optimization 
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approach for finding the minimal weight generators is described in Section |IVj In Section [V] we 
presents an LP union bound based on the generators' weight distribution. Section fVTI concludes 
the paper. 

II. MAPPING OF DECISION REGIONS 

The major difficulty of presenting the decision regions of a code longer than three is how to 
project or reduce an n-dimensional space to a two or three dimensional subspace. In this paper 
the n-dimensional space is sliced to a two-dimensional Euclidian subspace. A two dimensional 
subspace or a cut is a plane that is spanned by two orthogonal vectors. 

Consider transmitting an n-dimensional signal over an AWGN channel, such that the observed 
data is r = s + n where s is the transmitted signal, and n is a normally distributed noise with 
zero mean and variance a 2 . The decision regions {Z±, . . . , Zm} are the subsets of the signal 
space W 1 defined by 

Zi = {r : p( Si \r) > p( Sj \r) Vz ^ j, 1 < i,j < M} (1) 

where M is the number of codewords. 

The decision boundaries are all the points for which exists r6l" such that p(sj|r) = p(sj\r). 
The decision boundaries divide the signal space into M disjoint decision regions, each of which 
consists of all the point in R™ closest in Euclidian distance to the received signal r. An ML 
decoder finds which decision region Zi contains r, and outputs the corresponding codeword Cj. 
The existence of pseudocodewords in BP and LP decoders divides the signal space into more 
decision regions than those created solely by codewords. Clearly, these pseudocodewords reduce 
the decision regions of the codewords, hence deteriorate the decoder optimality. 

The first step towards mapping the decision regions is to decide of the two spanning vectors 
n,, and n x (n^iia. G W 1 ). These two vectors must be orthogonal in order to have a clear 2- 
dimensional picture, rather than a folded one. The sum of these two vectors is the noise vector 
that is added to the transmitted signal. The noisy samples are then decoded, and the output of 
the decoder is recorded along with the received signal. All the received signals which share the 
same output designate a decision region. In the following the LP decoder uses a C — symmetric 
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polytope [4J under a binary-input memoryless symmetric channel, thus one may assume without 
loss of generality that the all-zero codeword was transmitted. For BP and ML decoders, the 
conditional decoding error probability is independent of the codeword that was sent. Therefore, 
our analysis will assume that the all-zero codeword was transmitted over an AWGN channel 
using a BPSK modulation. The BP decoder being used is a sum-product decoder, configured to 
perform 50 decoding iterations. 

In LP decoding, the vertices of the fundamental polytope are a superset of the codewords. 
While the set of codewords are dominated by the code itself, the set of pseudocodewords is 
dominated by the relaxation being used. Let C be a binary code C E {0, l} n and let V(V) be 
the set of vertices of the polytope V. The polytope contains every codewords, but also some 
fractional pseudocodewords, thus: 

C C V(V) C?C[0, l] n . (2) 

The mapping of a vertex onto a point in a Euclidian plane is performed using the effective 
squared Euclidian distance [11] between a codeword c and a pseudocodeword p in a balanced 
computation tree: 

e//( °' p) = W\\ () 

where d = c — E[p] and a 2 = _E[||p|| 2 ] — \\E[p] || 2 . If the all-zero codeword is transmitted using 
a BPSK modulation, then © is simplified to 

d 2 eff (0,p)=Aw eff (p) (4) 
where w e ff is the effective Hamming weight in an AWGN channel, given by 

II II 2 (\^ n \2 

„.,AWGNC (r \ _ ... ( n \ — IIPIIl _ \L^i=\Vi) /n 
W p IPJ - W eff{P) = |j-j|2 - sr^n o " W 

IIPII2 l^i=\Pi 

Eq. © is sometimes referred to as the pseudoweight of p in an AWGN channel. The 
performance of iterative decoders is influenced mostly by the minimal weight pseudocodewords 
0, [TT2l . [TT3l . Ifl4l . while the ML decoder is influenced mostly by the code minimal Hamming 
weight. This is why the performance of an ML decoder is not affected by the selection of the 
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parity check matrix representation, which is not true for the case of BP and LP decoders. If all 
codewords are chosen with equal probability, then the effective Euclidian distance between the 
all-zero codeword and the decoded word is d e ff. The decision boundary between the all-zero 
codeword and the decoded word is exactly from the origin. 

The mapping of the decision regions within a two-dimensional cut is performed as fol- 
lows: 

l: Set the y-Axis to represent n y , and the x-Axis to represent n x 
2: Find the normalization factor for n y and n^: 



i=i 

3: for y in range min_added_noise to max_added_noise do 
4: for x in range min_added_noise to max_added_noise do 
5: Set the received signal 

r = Modulate^) + Uy ' V + ™ x ' - (7) 
norm_n y norm_n x 

6: Decode the signal 
7: end for 
8: end for 

9: Map the entire space spanned by y and x to the decoded words. 

Normalization of n y and is required in order to maintain a Euclidian space. A unit- 
step in the direction of n y is a step for which the noise that is added in the direction of 
n y will shift the transmitted signal by •y/^" =1 (n 2/ .) 2 from the origin. The min_added_noise 
and the max_added_noise are the two endpoints of both n y and n^, and are taken such that 

min_added_noise > — ^f 1 - and max_added_noise < d min , where d min is the code minimal 
Hamming distance. 

In the following the decision regions of the [8, 4, 4] extended Hamming code are studied. 
The [8, 4, 4] extended Hamming code is well-suited for studying the decision regions of the BP 
and LP decoders. It is a self-dual code which has a simple parity check matrix representation 
with minimal pseudoweight equals d min , but also a representation with pseudoweight equals 3. 



norm_n y = 



\ 



£K> 2 . 



norm_n x = 



\ 
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In order to gain a better understanding of the tight relation between the selected parity check 
matrix, the decision regions and the decoder's performance, three different parity check matrices 
are investigated ©, ®, ([TOi These matrices were originally introduced by Halford and Chugg 
in [fT5l , for which they also presented the pseudoweight spectra. 
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Fig. Q] presents the performance and the weight distribution of the three representations of the 



[8, 4, 4] extended Hamming code. Fig. 1(a) compares the frame error rate of the LP and BP 
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(b) Pseudoweight spectra 
Fig. 1. Different representations of the [8, 4, 4] extended Hamming code. 
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decoders using the parity check matrices of (8), (9) and (10). The performance difference between 
the LP and BP decoders is consistent for the 3 representations, in which the LP decoder slightly 
outperforms the performance of the BP decoder. Both decoders achieve the best performance 
when using H 3 and worst performance when using Hi. Furthermore, the LP decoder has almost 
the same performance as the ML decoder when using H 3 . 

Fig. | l(b)| illustrates the pseudoweight distribution of Hi, H 2 and H 3 . These results are similar 
to the ones presented in lfi~5l . A first observation is that Wp^^ NC (H 3 ) = d min = 4, which 
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(b) BP decoder 

Fig. 2. Decision regions of Hi with n y = (0, 2/3, 2/3, 2/3, 0, 0, 0, 0) and n x = (0,0,0,0,1,1,1,1). 
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(b) BP decoder 

Fig. 3. Decision regions of H 3 with n y = (0, 2/3, 2/3, 2/3, 0, 0, 0, 0) and n x = (0,0,0,0,1,1,1,1). 



can provide an explanation of why the suboptimal LP decoder is almost optimal. A second 
observation is that Hi has many more low-weight pseudocodewords compares to H 2 , which is 
consistent with the performance difference between the two representations. 

Fig. [2] and Fig. [3] illustrate some of the decision regions that were found by mapping the 
decision regions. The solid black lines represent the optimal decision regions of the ML decoder. 
The bottom left decision region represents the transmitted all-zero codeword. The decision region 
on its right (if any) represents another codeword which is a linear combination of and n x 
that lies in the same plane. The output of an ML decoder can only be a codeword; hence the 
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region above the solid lines is a region of codewords which are not a linear combination of n, y 
and n x . The soft output of the BP decoder enters a hard decision decoder to maintain a binary 
vector. 

Fig. [2] illustrates the decision regions in the plane spanned by n y = (0, 2/3, 2/3, 2/3, 0, 0, 0, 0) 
and n x = (0, 0, 0, 0, 1, 1, 1, 1). Clearly, the decision boundary between c and Ci obeys the 



d„ 



2 



rule. In the direction of n y , the decision boundary of the ML decoder is beyond since there 



is no competing codeword in this direction. The word p in Fig. 2(a) is a pseudocodeword, since 



it is fractional and has a weight equals 3 which is smaller than d min . Fig. |2(a)| also illustrates 
how the minimal- weight pseudocodeword p deteriorates the decoder's optimality by reducing 
the decision region of the transmitted codeword. 

Fig. |2(a)| and Fig. |2(b)| show that BP and LP decoders share the same decision boundaries 
between c and c 1? and same boundaries between c and p . The location of p in Fig. |2(a)| is 



c/ e// (0,po) = a/4w p (pq) = J 4 (2/ 3)2^ ( 2/3) l '2+( 3 2/3)2 = \/12 from c . The same calculation holds 
for Fig. |2(b)| 4//(0, p ) = a/4w p (p ) = J ^ m^+aVSi^ = The decision boundaries are 



_ _ C (l)2 + (1)2 + (1)2 

exactly at = = \/?> from c . Since the decision boundary is smaller than ^ Bl , the LP 
and BP decoders are not bounded distance ifTTl . ffi~8l decoders. The difference between BP and 
LP decoders is in the decision regions of pseudocodewords, and is caused due to the different 
algorithms that are used. While the decision regions of the LP decoder are convex polytopes [[Toll , 
their counterparts in BP decoding are non-convex and more chaotic. Clearly, the decision regions 
of p and pi in BP decoding are larger than their counterparts in LP decoding. Nevertheless, 
from this figure it is clear that this difference has no major impact on the performance. 

Fig. [3] represents the decision regions when using H 3 and the same n,, and n x as in Fig. [2] 
From Fig. |3] one can observe that H 3 does not contain the (0,2/3,2/3,2/3,0,0,0,0) pseu- 
docodeword, but rather the higher-weight pseudocodewords (0, 2/3, 2/3, 2/3, 2/3, 2/3, 2/3, 2/3) 
and (0,2/3,2/3,2/3,1/3,1/3,1/3,1/3) of weight 7 and 6.25, respectively. In this case the 
pseudocodewords barely reduce the decision region of the transmitted codeword, which explains 
why H 3 defines a polytope which makes the LP decoder almost optimal. One can observe that p 
and pi in Fig. |3(b)| reduce the optimal decision region of c slightly more than their counterparts 
in Fig. |3(a)[ but still maintain a bounded distance decoding. This observation is correlated with 
the actual performance of the two decoders. The position of BP pseudocodewords in signal 
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space is sometimes misleading, due to the information loss caused by the hard decision at the 
output of the BP decoder. This is why we omitted the position of p and p x from Fig. |3(b)| The 
fact that the decision regions of p and p 1 in BP decoding are much larger than those of the 
LP decoder does not necessarily reflected in the performance, since the majority of the area is 
located outside the optimal decision region of c , i.e. in the error region. 

III. The effect of minimal weight pseudocodewords on lp decoding 

Simulations show that although the performance of the LP decoder is dominated by low- 
weight pseudocodewords, not all low-weight pseudocodewords have the same contribution to 
the error probability. In this section we will justify why some minimal weight pseudocodewords 
may have a higher contribution to the error probability compared to others. We will base our 
justification on both simulation results and decision regions. 

The polytope of Hi has 26 vertices of pseudoweight equals 3. Our simulation results for 
different SNRs show, for example, that the pseudocodeword p = (0,2/3,2/3,2/3,0,0,0,0) 
causes approximately 4 times more decoding errors than p x = (1, 0, 0, 0, 0, 1/3, 1/3, 1/3) . There 
are several properties that affect the error probability of a given pseudocodeword. When listing 
all pseudocodewords, one can see that there are many pseudocodewords with support equals 
4 that share 3 out of 4 non-zero components with p x . It means that in the objective function 
the selection between such candidates depends on two independent random variables. However, 
there are no pseudocodewords with support equals 3 that share 2 components out of 3 with p . 
There are pseudocodewords with support higher than 3 that contain non-zero components in the 
same positions as p but with lower values. Such components have weaker effect on the cost 
value, and lead to fewer decoding errors. 

We will now present a cut that contains the aforementioned p and p 1? and show that in 
the specific cut the decision region of po is larger than that of pi, which provide another 
perspective of why p causes more decoding errors. Fig. |4] presents a cut created by the noise 
vectors n y = (0,2/3,2/3,2/3,0,0,0,0) and n, = (1,0,0,0,0, 1/3, 1/3,1/3). Notice that the 



decision regions, at least in the presented cut, have a different behavior as illustrated in Fig. |4(a) 
While the decision region of pi increases only in the y-axis, the decision region of p grows in 
both axes. 
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Fig. 4. Decision regions of an LP decoder with n y = (0, 2/3, 2/3, 2/3, 0, 0, 0, 0) and = (1, 0, 0, 0, 0, 1/3, 1/3, 1/3). 
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The existence of p and pi in the polytope of Hi reduces the decision region of c as shown 



in Fig. |4(a)| The polytope of H 2 does not contain p ls thus the decision region of c is bigger 
than that of Hi as presented in Fig. |4(b)| The fundamental polytope of H 3 does not contain p , 
nor pi, thus the decision region of c is identical to that of the optimal ML decoder, as can be 



seen from Fig. 4(c) 



IV. Finding the Minimal Weight Generators 



Inspired by the work of [ 1161 we were encouraged to seek for a deterministic approach for 
finding the minimal weight generators. The heuristic method of ffT6 1 provides an excellent upper 
bound on the minimal pseudoweight, and it can be used for long and dense codes. It still lacks 
the certificate that the minimal weight generator that was found using this method is the minimal 
weight generator of a given parity check matrix. The number of iterations that are needed to 
reach a tight bound is also left open. The number of minimal weight generators is fundamental 
for obtaining a union bound, but the method of [16] can only estimate this number. 

In this section we present a method for finding the minimal weight generator of a given parity 
check matrix, as well as the number of minimal weight generators. 



Proposition 1. ([9]) The pseudoweight is invariant under scaling. 



Proof: We need to prove that w. 



AWGNC 
V 



(ap) 



AWGNC, 



p) where a is a real positive 



number, and p € 



is a pseudocodeword. 
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Given a binary parity check matrix H e F™ xn , the fundamental cone /C = JC(H) is defined 
as the conic hull of the fundamental polytope P(H), and can be described by the following set 
of linear inequalities: 

Pi >0 and 

JC=\peR n J2ti,i&> h jiPi > h ji'Pi' > < 12 ) 
V 1 < « < n, V 1 < j < m 

where hp denotes the entry of H in the jth row and ith column. 
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p e 



(13) 



Corollary 2. AZZ points on an edge of the fundamental cone have the same pseudoweight. 

Proof: Let a and b be two points on the edge e fC. From the definition of K, it is clear 
that each edge is a ray with an endpoint at the origin. Being on the same ray, one can express 
a = ah where a is a real positive number. From Proposition^ it follows that a and h have the 
same pseudoweight. ■ 

Searching for the minimal weight generator is equivalent to searching for the minimal weight 
edge of the fundamental cone K,. We will now prove that for searching for the minimal weight 
edge one can bound /C and use the following polytope: 

Pi > and 

YJ!=iPi = a > a > 

EILwi h HPi > h ii'Pi' 
VI < % < n, VI < j < m 

where the essence of the constant a E M is to increase the dynamic range of the problem and 

prevent scaling issues in optimization softwares. 

Proposition 3. The edges of fC and the vertices of have the same weight distribution. 

Proof: Let Ei be an edge of K, and let p be an arbitrary point on Ei. From H~3]) it is clear 
that Ei is also an edge of Vk- Suppose Yl7=i Pi = ^> an< ^ ^ et ^ = % then according to 071) 
the pseudocodeword p' = kp has the same weight as p and the sum of its components is a; 
Thus p' is a vertex ofVK- Now, let Ej be an edge ofVK- From the definition ofVK, Ej has two 
endpoints: One in the origin and one in p for which Y^h=i P* = a - Clearly, JC has an edge with 
an endpoint in the origin, that passes through p and goes off to infinity. According to Corollary 
|2] the pseudoweight of this edge in JC is the same as the vertex p G Vk, which completes the 
proof. ■ 

The problem of finding the minimal weight in an AWGN channel becomes: 



w,. 



mm w p (p) 



mm 



mm 



(14) 



January 20, 2013 



DRAFT 



16 



where the last equation follows from the definition of Vjc. Being a constant, a does not affect 
the minimization process, thus instead of solving (fl4l) one may consider solving the following 
simpler maximization problem: 

n 

maxVVpj) 2 . (15) 

The minimal pseudoweight w Pmin is simply the division of a 2 by the optimal solution of (fl"5l) . 

The maximization problem of (fl3T > is non-convex and may have several local maxima. Algo- 
rithms for solving such problems are termed Global Optimization and are able to find the global 
solution in the presence of multiple local solutions. 

Global optimization algorithms are usually divided into deterministic and probabilistic ap- 
proaches. The solution of a deterministic approach is guaranteed to be the global solution, or 
at least a local solution which differs from the global solution by less than a given e > 0. 
Probabilistic algorithms require a shorter runtime compares to deterministic ones, but their 
solution may not be the global optimum. 

An efficient deterministic approach for solving global optimization problems is the Branch 
and Bound algorithm. The algorithm relies on the existence of a convex relaxation of the original 
problem |fl9l , whose optimal solution provides a lower bound on the solution of the original 



Code 


w Pmi „ 


N Pmin 


Hamming [15, 11, 3] 


3.0 


127 


Hamming [31, 26, 3] 


3.0 


1185 


BCH [31, 21, 5] 


3.0 


6 


BCH [63, 45, 7] 


3.0 


1 


BCH [63, 39, 9] 


3.299176 


1 


BCH [63, 36, 11] 


3.2 


83 


BCH [127, 113, 5] 


3.0 


134 


BCH [127, 64, 21] 


3.0 


2 


BCH [255, 131, 37] 


3.33333 


9 


Tanner [155, 64, 20] 


16.403683 


465 



TABLE I 

Minimal-weight and number of minimal- weight generators 
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problem. 



The simplest probabilistic global optimization algorithm is the Multistart algorithm, which uses 
a local algorithm starting from several points distributed over the whole optimization region. The 
local optimum with the best objective value is taken as the global solution. 

Some global optimization softwares such as BARON (Branch And Reduce Optimization 
Navigator) 11201 can also provide the fc-best solutions, or all local solutions, which guarantee 
not only finding the minimal weight, but also the distribution of the minimal weight generators. 

Being deterministic and efficient, BARON was chosen as the global optimization software. 
The minimal weight along with the number of minimal weight generators N Pmin for several 
selected codes are presented in Table U We performed short cycles reduction [fl"5l on the BCH 
codes to improve their performance under iterative decoding. The minimal pseudoweight for the 
[155, 64, 20] Tanner code [21] presented in [fT2~l is dip ~ 16.4037, which is similar to our 
results. Notice that the LDPC Tanner code has a minimal pseudoweight much higher than all 
the tested HDPC codes. Although short cycles were removed from the BCH codes, we were not 
able to increase the minimal pseudoweight beyond 3~. It is interesting to develop new methods 
for increasing the minimal pseudoweight of a given dense parity check matrix without adding 
redundant parity checks. 



A union bound for LP decoding was mentioned in lfl4l . 11221 and 11231 . but a full characterization 
of such a bound was not provided. In this section we examine the [8, 4, 4] Extended Hamming 
code and the [24, 12, 8] extended Golay code, and present why calculation of such a bound is 
not an easy task. 

The ML union bound [|24l for the case where the all-zero codeword s is transmitted is 



where M is the number of signals and d 0i is the distance between s and Sj. Clearly, (fl~6l) can 
be very loose in case the individual events are not disjoint. For large SNRs the union bound of 
(fl~6l) can be approximated by including only the dominating terms: 



V. LP Union Bound 




i=l 



(16) 
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Pr [error | s ] « iV min Q 



2a 



(17) 



where N min is the number of nearest neighbors of the transmitted signal s . We can no longer 
assume that (fTTT) is an upper bound, since we've neglected positive terms from (TT6T ). 

Assuming that the all-zero codeword is transmitted, the error probability over the fundamental 
polytope is equal to that over the fundamental cone A3; Thus, a union bound for the LP decoder 
can be formulated from (fTol as follows: 



Pr[error\s ] < Q ( 
i=i 



2a 



(18) 



where N p is the number of generators and w Pi is the pseudoweight of generator i. Similarly, we 
can obtain an approximation for large SNRs: 



Pr [error \ So transmitted] ~ N Pmin Q 



2a 



(19) 



where N Pmin is the number of minimal weight generators, and w Pmin is the minimal pseudoweight. 

In the following we will use the parity check matrices of ©, © and (ITQb for the Extended 
[8, 4, 4] Hamming code and (|20|) . (|2TT) for the [24, 12, 8] extended Golay code. 



Hr 



100110101111000001010011 
110011010111100000101001 
011001101011110000010101 
001100110101111000001011 
100110011010111100000101 
010011001101011110000011 
101001100110101111000001 
010100110011010111100001 
001010011001101011110001 
000101001100110101111001 
000010100110011010111101 

111111111111111111111111 



\ 



(20) 
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/ 



H, 



G' 



\ 



\ 



(21) 



/ 



100110101111000001010011 
010010001 100010000101001 
1 1 10001 10000000000010101 
000100100001101010000110 
10000101 101 1 1 10000000000 
000001000001001110101010 
101001100110101111000001 
001101011000100111110100 
0001 1 1000001001 100000101 
001000010100010010001 101 
001010110010001000110000 
110101100110010100001110 
The parity check matrices for the [24, 12, 8] extended Golay code were introduced by Halford 
in 11251 . The parity check matrix in (1211 was obtained by applying short cycles reduction [fT51 
on d20l) . The Tanner graph representation of H G contains 1,551 4-cycles and 65,632 6-cycles, 
while there are 295 4-cycles and 6,204 6-cycles in the representation of H G i. It was shown in 
11251 that the message passing algorithm using H G > outperforms the one using H G by more than 
1 dB. 

From Section |nll it is clear why having a tight LP upper bound is a complicated task: In the low 
SNR regime the bound may be very loose if not scaling each minimal weight pseudocodeword 
by its contribution to the error probability. In the high SNR regime, especially for medium and 
long codes, the minimal weight pseudocodewords may have low-volume decision regions, thus 
may have a negligible effect on the performance of the LP decoder. A similar phenomenon was 
observed in [fTTl . lfT8l for pseudo nearest neighbors in bounded-distance decoding algorithms. 
In [fT8l Fishier et al. derived an approximated probability ratio between the error contribution 
of a non-codeword neighbor and a codeword nearest neighbor. The ratio was calculated based 
on the ratio between the volumes of the decision regions of the two competitive neighbors. 
Incorporating this ratio yielded a better approximation for an upper bound. 
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(a) [8, 4, 4] extended Hamming code 
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(b) [24, 12, 8] extended Golay code 
Fig. 5. Weight distribution of the [8,4,4] extended Hamming code and the [24, 12,8] extended Golay code. 
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Fig. \5\ presents the generators' weight distribution of the [8, 4, 4] extended Hamming code and 
the [24, 12, 8] extended Golay code. Clearly, using ([TBI as an upper bound for the LP decoder 
will result an error probability much higher than unity, which makes the bound useless. The 
reason is that the large number of generators leads to many joint events, which make the bound 
very loose. Strictly speaking, while the [24, 12, 8] extended Golay code has 4096 codewords, 
there are 91,113,330 and 231,146,334 generators for H G and H G >, respectively. It was mentioned 
above that H G ' was obtained from H G by applying short cycles reduction. Notice that not only 
Hq> has a higher minimal pseudoweight, but its entire pseudoweight spectra is centered to the 
right of the one of H G , as illustrated in Fig. |5(b)[ 



Code 


Parity Check Matrix 


w P mi „ 


N Pmin 


Extended Hamming [8, 4, 4] 


Hi 


3.0 


26 


H 2 


3.0 


4 


H 3 


4.0 


14 


Extended Golay [24, 12, 8] 


H G 


3.0 


2 


H g > 


3.6 


2 



TABLE II 

Minimal-weight generators of the [8, 4, 4] extended Hamming code and the [24, 12, 8] extended Golay code 



Table [XT] presents the number of minimal-weight generators of the aforementioned parity 
check matrices. The pseudoweights from Table [XT] were used to calculate the LP Union Bound 
Approximation (UBA) of ( fl9t . The results are presented in Fig. |6] For clarity the actual error 
probabilities are also presented. The performance of the LP decoder for the chosen matrices are 
correlated with the generators' weight distribution that was presented in Fig. |5] From Fig |6(a) 



one can see that the suggested UBA is tight for Hi and H 3 , but inaccurate for H 2 . The UBA 
is much worse for the case of the extended Golay code, as presents in Fig. |6(b)[ In this case 
the UBA does not reflect the actual behavior of the decoder, but rather presents a too-optimistic 
behavior. The reason is due to the fact that there are only two minimal-weight generators for 
both representations, whereas the ML UBA (fT7l employs 759 nearest neighbors. Notice that 
there are 91,113,326 and 230,918,045 generators of pseudoweight lower than d min for H G and 
He, respectively. Had we considered all the generators having pseudoweight lower than d m i n 
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■ ♦ - H3 LP UBA 
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(a) [8, 4, 4] extended Hamming code 




(b) [24, 12, 8] extended Golay code 
Fig. 6. LP UBA for different representations of the [8, 4, 4] extended Hamming code and the [24, 12, 8] extended Golay code. 
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in ( fT9l ), we would have gained an LP UBA higher than unity. The LP UBA does not have a 
coherent behavior, i.e. the bound can be sometimes below the actual performance curve, which 
again disqualifies it as an upper bound or as an approximation. 

The pseudoweight and the weight-distribution are not enough for implementing a tight LP 
upper bound. A tight and accurate bound must take the volume of the decision regions into 
account. As was presented, some low-weight pseudocodewords which have small volume have 
also small effect on the decoder's performance, but are very dominant in the equation of the LP 
union-bound. A tight LP union-bound must have a distinction between codewords and different 
types of pseudocodewords. 

VI. Conclusion 

The decision regions of the LP decoder were studied and compared to those of the BP and 
ML decoders. We showed that both BP and LP are not bounded distance decoders. The different 
effect of minimal-weight pseudocodewords on the performance of the LP decoder was examined. 
Global optimization was presented as a method for finding the minimal pseudoweight, as well 
as the number of minimal weight generators. An LP union bound was presented, along with an 
explanation of why having the pseudoweight spectra is not sufficient for finding a tight bound. 
Finding the ratio between the error contribution of a nearest pseudocode word and a nearest 
codeword will tighten the union bound, and is left open for further research. 
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